Uav-based sensing for worksite operations

ABSTRACT

A sensor senses an attribute of a worksite at a location that is geographically spaced from a corresponding mobile machine. An operation is performed at the location, based upon the sensed attribute. An action signal is generated based on the effect data. An unmanned aerial vehicle communicates effect data, indicative of an effect of the operation at the location, to the mobile machine. The action signal can be used to control worksite operations.

FILED OF THE DESCRIPTION

The present description deals with worksite operations. Morespecifically, the present description deals with using an unmannedaerial vehicle (UAV) to gather data for use in controlling worksiteoperations.

BACKGROUND

There are a wide variety of different types of mobile machines. Thesetypes of machines can include agricultural machines, turf care machines,forestry machines, construction machines, etc. The machines are used inperforming a wide variety of functions in worksite operations.

One example includes agricultural machines. Site-specific farming refersto performing crop care functions, only where needed within a field.Therefore, some work has been done in sensing attributes of a field, andcorrelating them with geographic location, in order to generate mapsbetween sensed attributes and their location in a field. Some suchsystems sense attributes in a field by using remote images that arecaptured by either aircraft or satellite platforms. Other approacheshave used cameras on ground-engaging machines to capture images.

The discussion above is merely provided for general backgroundinformation and is not intended to be used as an aid in determining thescope of the claimed subject matter.

SUMMARY

A sensor senses an attribute of a worksite at a location that isgeographically spaced from a corresponding mobile machine. An operationis performed at the location, based upon the sensed attribute. Anunmanned aerial vehicle communicates effect data, indicative of aneffect of the operation at the location, to the mobile machine. Anaction signal is generated based on the effect data. The action signalcan be used to control worksite operations.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. The claimed subject matter is not limited to implementationsthat solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one example of a crop care machinearchitecture.

FIG. 2 is a more detailed block diagram of one example of the crop caremachine architecture shown in FIG. 1.

FIG. 3 is a block diagram of one example of another crop care machinearchitecture.

FIG. 4 is a flow diagram illustrating one example of the operation of acrop care machine architecture with an unmanned aerial vehicle operatingforward of a mobile crop care machine.

FIG. 5 is a flow diagram illustrating one example of the operation of acrop care machine architecture with an unmanned aerial vehicle operatingrearward of a mobile crop care machine.

FIG. 6 is a block diagram of one example of a crop care controller, inmore detail.

FIG. 7 is a flow diagram illustrating one example of the operation ofthe crop care machine architecture shown in FIG. 3, in more detail.

FIG. 8 is a block diagram illustrating the operation of a crop caremachine architecture in using a difference map.

FIG. 8A shows one example of sensed values that can be used to generatea metric.

FIG. 9 is a block diagram illustrating one example of a more detailedimplementation of the crop care machine architecture shown in FIG. 3.

FIG. 10 (which includes FIGS. 10A, 10B and 10C) shows patterns that canbe detected to identify errors or malfunctions.

FIG. 11 is a block diagram of one example of a calibration path.

FIG. 12 is a pictorial diagram of one example the crop care machinearchitecture shown in FIG. 3, depicting a docking area.

FIG. 13 shows one example of how information can be used in a remoteserver architecture.

FIGS. 14-16 show examples of mobile devices that can be used inarchitectures shown above.

FIG. 17 shows an example of a computing environment that can be used inthe architectures shown in previous Figures.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of one example of a crop care machinearchitecture 100. It will be appreciated that architecture 100 could bean architecture in which another kind of machine is used, such as amachine used in construction, turf care, forestry, or anotheragricultural machine, etc. It is described herein as a crop care machinearchitecture for the sake of example only.

Architecture 100 illustratively includes a crop care machine 102 and anunmanned aerial vehicle (UAV) 104. UAV 104 illustratively includes apropulsion mechanism 106, such as one or more rotors that are driven byone or more motors. In the example shown in FIG. 1, UAV 104 is coupledto mobile crop care machine 102 by link 108. In one example, link 108 isa mechanical tether that provides a power link from machine 102 to UAV104, in order to power UAV 104. Link 108 also illustratively provides acommunication link between machine 102 and UAV 104 so that UAV 104 canprovide sensor data (indicative of sensed values that are sensed bysensors on UAV 104) back to machine 102. It can also illustrativelyprovide geographical information to indicate the relative positions ofmachine 102 and UAV 104, and it can carry control signals by which acontroller on machine 102 can control UAV 104, or vice versa.

In one example, machine 102 travels over a worksite (such as a field).UAV 104 illustratively flies in proximity to machine 102, sensing one ormore attributes of the worksite. Attribute indicators, that indicate thesensed attributes, are provided to machine 102, so that machine 102 cancontrol its own operation (or the operation of other machines) based onthe sensed attributes. Also, any number of metrics can be calculatedbased on the sensed attribute and provided to other machines or otheranalysis systems.

It will be noted that, while some of the present discussion proceedswith respect to machine 102 being a sprayer for applying chemicals to acrop or field, machine 102 can be any of a wide variety of differenttypes of machines. For instance, it can be a planter, a harvester, atillage machine, or a wide variety of other machines. It will also benoted that, in architecture 100, UAV 104 can fly in a forward directionrelative to machine 102 and send back data that can be used by machine102 to perform its operations. In another example, UAV 104 can flybehind machine 102 and send information to machine 102 indicative of howthe operation performed by machine 102 was actually performed (e.g.,indicative of the quality of the operation performed by machine 102). Inyet another example, UAV 104 can fly both forward of, and rearward of,machine 102, alternately. Thus, for instance, UAV 104 can first flyforward of machine 102 sensing a given attribute of the crop or fieldand providing that information to machine 102 so that machine 102 canadjust its operation. It can then fly rearward of machine 102 to sensehow well machine 102 actually performed its operation, and provide thatinformation back to machine 102. Machine 102 can adjust its operation,if needed. In yet another example, there are multiple UAVs 104 linked tomachine 102 so that some can fly forward of machine 102, some can flyrearward of machine 102, etc.

Before describing any of these examples in more detail, a more detaileddescription of one example of machine 102 and UAV 104 will first beprovided. FIG. 2 is a block diagram illustrating one example of the cropcare machine architecture 100 (shown in FIG. 1) in more detail. In theexample shown in FIG. 2, UAV 104 can have propulsion system 110 thatdrives rotors 106, one or more attribute sensors 112, communicationcomponent 114, positioning system 116, and a set or processors orcontrollers 118. It can have other items 120 as well. Processors orcontrollers 118 can include a propulsion control component 122, sensorcontrol component 124, communication control component 126, and it caninclude other items 128.

The example shown in FIG. 2 illustrates that mobile crop care machine102 can have its own propulsion system 130, one or more user interfacecomponents 132, and communication system 134 (which can include UAVcommunication component 136 and other communication components 138).Machine 102 can have crop care control system (or controller) 140 (whichcan output control signals to control a set of controllable crop caremechanisms 142) positioning system 144, a variety of other sensors 145,power system 146, and it can include other items 148). Before describingthe overall operation of architecture 100 in more detail, a briefoverview will first be provided.

Again, mobile crop care machine 102 will be described in some examplesas a sprayer. However, crop care machines can include a wide variety orother machines as well. They can include, for instance, self-propelledsprayers, towed sprayers, mechanical weeders, laser weed killers,fertilizer applicators, planting machines, harvesters, and a widevariety of other machines.

Therefore propulsion system 130 can be any propulsion system which issuitable to the particular machine. Power system 146 can be a systemthat provides power to machine 102 and also to UAV 104. However, machine102 can also be powered by a separate power system from UAV 104. In oneexample, for instance, it includes an engine with a transmission thatdrives ground-engaging mechanisms (such as wheels, tracks, etc.). Userinterface components 132 can be a wide variety of user interfacecomponents that allow a user to interface with the other portions ofmachine 102. For instance, they can include levers, switches, wheels,joysticks, buttons, a steering wheel, pedals, etc. They can also includemicrophones with speech recognition systems and natural languageprocessing systems, to process speech inputs. They can include userinput mechanisms that can be actuated from a user interface display. Forinstance, they can be icons, links, drop down menus, radio buttons, textboxes, or a wide variety of other user input mechanisms that can beactuated by a user, on a user interface display screen. The user inputmechanisms can be actuated to control and manipulate mobile crop caremachine 102 or UAV 104, or both.

Crop care controller 140 illustratively receives information from UAV104, and it can receive information from a wide variety of the othersensors 145. It then generates a control signal to control controllablecrop care mechanisms 142. Crop care mechanism control may includeadjusting frame or ground engaging element height above ground; groundengaging element depth into or below ground, downpressure, or angle;chemical application type or rate; chemical application pattern; orother mechanism control. It can be used to control machine speed aswell. In various other applications, controller 140 can controlmechanisms 142 to perform a wide variety of other operations as well.Some of them include boom height, nozzle size, solution delivery systems(flow and pressure), depth control, downforce systems, surface finish ofthe soil (e.g., is it level and smooth), cutter height, residuedistribution system (even spread of residue behind harvester byincreasing spinner speed), row cleaner height (residue removal), augerpositions, spout positions, machine-to-machine position, vehicletraction control system, vehicle and implement steering control, amongothers.

Positioning system 144 illustratively generates a position indicatorindicating a position of machine 102. For instance, it can be a globalposition system (GPS), a dead reckoning system, a cellular triangulationsystem, or a wide variety of other positioning systems.

On UAV 104, propulsion system 110 illustratively powers rotors 106, orother mechanisms to provide propulsion to UAV 104. Propulsion controlcomponent 122 illustratively controls propulsion system 110. In doingso, it can illustratively control the direction, height, attitude,speed, and other characteristics of UAV 104.

Attribute sensors 112 illustratively sense one or more attributes of afield or a crop over which machine 102 is traveling. For instance,attribute sensors 112 can sense such things as soil, soil type, soilmoisture, soil cover, residue density, crop type, weed presence and weedtype, plant size, plant height, plant health, plant vigor, chemicalpresence, chemical distribution, etc. Sensors 112 can thus be a widevariety of different types of sensors, such as cameras, infrared camerasor other infrared sensors, video cameras, stereo cameras, LIDAR sensors,structured light systems, etc.

Sensor control component 124 can illustratively control attributesensors 112. Therefore, it can illustratively control when sensorreadings are taken, and it can perform signal conditioning on the sensorsignals, such as linearization, normalization, amplification, etc. As isdescribed below, it can also illustratively perform other processing onthe attribute signals, or that processing can be performed by crop carecontroller 140 on machine 102, or the processing can be split betweencomponent 124 and component 140.

Communication component 114 illustratively communicates with mobile cropcare machine 102. It can communicate by a wired communication harnesswhen link 108 is a physically tethered harness. It can also communicatethrough a wireless communication link. Communication control component126 illustratively controls communication component 114 to communicatewith machine 102. It can communicate the attribute sensor signals fromsensors 112, it can communicate them after they are conditioned bycomponent 124. It can also communicate other values that are generatedbased on the attribute sensors, or other items in UAV 104. For instance,it can communicate the position of UAV 104 identified by positioningsystem 116. It can also, for example, calculate a relative offsetbetween the position of UAV 104 and the position of machine 102, andcommunicate that value to machine 102. It can control the communicationof a wide variety of other values or signals between UAV 104 and machine102 as well.

Positioning system 116 illustratively generates a position indicator,indicating a position of UAV 104. As with positioning system 144, system116 can be a GPS system, a cellular triangulation system, a deadreckoning system, or a wide variety of other types of systems.Positioning system 116 may also generate pitch, roll, or yaw indicatorswhich indicate pitch, roll, or yaw of UAV 104.

It will also be noted that the various processing of items mentionedwith respect to FIG. 2 can be performed in other locations (instead ofUAV 104 or machine 102) as well. For instance, various signals or valuesmay be transmitted to a remote location where the processing isperformed. The results of the processing may be stored at the remotelocation or returned to architecture 100 for use within architecture100, or both.

Again, before, discussing the operation of architecture 100 in moredetail, another implementation of architecture 100 will first be brieflymentioned. FIG. 3 shows one example of architecture 152. Architecture100 may be an implementation of architecture 100, or a differentarchitecture. It can be seen in FIG. 3 that architecture 152 includesmobile crop care machine 102, UAV 104, and link 108. FIG. 3 alsoillustrates that machine 102 has a controllable crop care mechanism 142that performs an operation on a portion 157 of the worksite, as machine102 moves over the worksite in the direction indicated by arrow 158.Architecture 152 also illustratively includes another UAV 154 which canbe coupled to machine 102 by link 156. Thus, in the example shown inFIG. 3, machine 102 illustratively travels across a field or worksite ina direction indicated by arrow 158. UAV 104 illustratively flies forwardof machine 102, and UAV 154 illustratively flies rearward of machine102. UAV 104 illustratively uses the one or more attribute sensors 112disposed thereon in order acquire worksite data from a portion 160 ofthe worksite. UAV 154 illustratively uses its sensor to acquire worksitedata from a portion 162 of the worksite, after machine 102 has passedover it.

It will of course be noted that, for UAV 104 to acquire worksite datafrom portion 160 of the worksite, it may need to fly back and forth in adirection generally perpendicular to the direction of travel 158, infront of machine 102. In another example, attribute sensors 112 may havea sensing field which is wide enough to cover portion 160. It yetanother example, multiple UAVs 104 are used to acquire data from portion160 ahead of machine 102.

In one example, the same is true of rearward UAV 154. Data can beacquired from portion 162 by UAV flying back and forth behind machine102, or by deploying multiple UAVs behind machine 102. It can alsoacquire data with sensors that have a sensing field that is wide enoughto cover portion 162.

In yet another example, a single UAV 104 is used that intermittentlyflies forward of machine 102, to acquire data from portion 160, and thenrearward of machine 102 to acquire data from portion 162. All of theseimplementations are contemplated herein.

In any case, portions 160 and 162 generally have a dimension D that issimilar to the corresponding dimension D of portion 157, that is beingoperated on by controllable mechanism 142. Therefore, as is described ingreater detail below, UAV 104 can acquire data with respect to portion160 and communicate that data back (and its location) to machine 102.Machine 102 can use the data to control the operations performed bycontrollable crop care mechanism 142 in portion 157, when machine 102has moved forward in the direction 158 far enough that portion 157 isthe same as portion 160. UAV 154 can then acquire data from portion 162(again when portion 162 is the same as portion 157) to indicate thequality of the operation performed by controllable crop care mechanism142. This can again be transmitted back to machine 102 which can performa wide variety of different operations based on that information.

As but one concrete example, machine 102 can be a sprayer. The sprayermay be a direct injection sprayer that is capable of spraying severalagricultural chemicals in different concentrations at the same time.Those chemicals may further include a marker chemical such as a dye,which is detectable by rearward UAV 154, using attribute sensors 112.UAV 104 can illustratively capture images of portion 160 in a field thatmachine 102 is traveling over. It can transmit that information overlink 108 to machine 102. Crop care controller 140 can then identify thelocation and types of weeds in those images. It can control individualspray nozzles to dispense different types of chemicals, at differentlocations, to treat the weeds identified in portion 160, when sprayer142 is over portion 160. UAV 154 can then identify whether the sprayerapplied chemicals to the appropriate locations when it passes over theportion of the field where the chemicals were sprayed. In someinstances, this is accomplished by sensing the level, presence, orabsence of marker on the target locations such as on weed leaves. Ofcourse, this operation can be continuous so that for each location inthe field, the data is transmitted from UAV 104 to machine 102 and thenfrom UAV 154 to machine 102, and correlated geographically, as machine102 travels over the field. The operation can be performedintermittently as well.

The operation of architectures 100 and 152 will now be described withrespect to a variety of different implementations. First, for instance,FIG. 4 is a flow diagram which illustrates the operation of architecture100, when only a forward UAV 104 (or a set of forward UAVs) are deployedwith respect to machine 102. In such an implementation, there is norearward UAV 154.

UAV 152 thus first uses its one or more attribute sensors 112 to sensean attribute of an area of the worksite that is forward of the mobilecrop care machine 102, as machine 102 is traveling in the forwarddirection. This is indicated by block 166 in FIG. 4. In doing so, UAV104 can correlate the attribute signal with a position from which it issensed, as indicated by positioning system 116.

UAV 104 then communicates an indication of the sensed attribute, and thelocation that it was sensed from, to mobile crop care machine 102 overlink 108. This is indicated by block 168. Crop care controller 140 onmachine 102 then processes the sensed attribute indicator to generate anaction signal. This is indicated by block 170. For instance, whenmachine 102 is a sprayer, crop care controller 140 can generate aspraying prescription that prescribes one or more chemicals (such as thechemical type, the chemical concentration and its location ofapplication).

Crop care controller 140 then illustratively controls the controllablecrop care mechanisms 142, based upon the action signal. This isindicated by block 172. For instance, using the spraying prescription,and the location information provided by positioning system 144, cropcare controller 140 can control individual actuators (such as spraypumps, valves, nozzles, etc.) on mechanism 142 in order to applychemicals to conform to the prescription that was provided for thecurrent geographic location of the sprayer. It can thus controlmechanisms 142 to apply a particular chemical, of a particular chemicaltype, in a particular chemical concentration, at a desired location ofapplication.

In some examples, attribute sensor 112 may be a camera or another imagecapture mechanism that captures an image of the portion 160 of the fieldor worksite. Images may be two dimensional or three dimensional. Imagesmay be from electromagnetic radiation reflected by, emitted from, ortransmitted through, an object. Images may comprise alone or incombination electromagnetic radiation intensity, wavelength, band timeof flight, phase shift, or any other suitable image parameter. In suchan implementation, crop care 140 illustratively processes the image toidentify weeds. The image processing may use a wide variety of differenttypes of techniques, such as leaf spectral reflection characteristics,shape (or morphology) or other features to identify crop and weedspecies. Plant size may be estimated using pixel width in field of view,stereo imaging, time-of-flight reflectance, structural light, or otherprocesses or mechanisms. These or other parameters can be used ingenerating the prescription, by selecting the chemical type andconcentration (or dosing). In such an implementation, the chemicals maybe different types of herbicides.

Also, in one implementation, the chemical concentration may vary acrossthe width of the area being treated by machine 102. It may also varywith the distance traveled by machine 102.

In another example, the image captured by UAV 104 may be processed todetermine plant health or vigor. Instead of the prescription being toapply a herbicide, the chemicals may comprise nutrients, such asnitrogen, phosphorus, potassium, micro-nutrients, such as sulfur, iron,etc., soil pH modifiers, such as lime, among a wide variety of otherchemicals.

In still another example, the image can be processed to determine thepresence and severity of pests and diseases. In that case, theprescription can be to prescribe chemicals that may include pesticides,insecticides, fungicides, nematicides, etc. FIG. 5 is a flow diagramillustrating one example of the operation of architecture 100, whereonly a rearward UAV 154 is deployed. In such an implementation, there isno forward UAV 104.

In the implementation described with respect to FIG. 5, UAV 154 firstsenses an attribute of an area of a worksite that is rearward of mobilecrop care machine 102. This is indicated by block 174 in FIG. 5. It thencommunicates an indication of the sensed attribute, and its location, tomachine 102. This is indicated by block 176. Crop care controller 140then processes the sensed attribute indicator to generate a metricindicative of a quality of crop care operation performed by the mobilecrop care machine 102 (and controllable mechanism 142) at thatgeographic location. This is indicated by block 178. Controller 140 canoutput the metric for use in various ways. A number of the ways will bedescribed in greater detail below. Outputting the metric for use isindicated by block 180.

In the implementation described with respect to FIG. 5 (again where theexample of machine 102 is a sprayer), the prescription may be an apriori prescription, or it may be derived from sensor data that isobtained from vehicle mounted workspace sensors 145. In such animplementation, rearward UAV 154 is used to monitor the quality ofapplication of the chemical, and optionally to perform touch-up spraying(as is described in greater detail below).

Before describing yet another implementation, in which both forward UAV104 and rearward UAV 154 are present, a more detailed description of oneexample of crop care controller 140 will first be provided. FIG. 6 showsa more detailed block diagram of one example of crop care controller140. In the example shown in FIG. 6, controller 140 illustrativelyincludes geographical correlation component 190, prescription generatorcomponent 192, and difference map generation component 194. It caninclude pattern identifier component 196, supplemental informationcollection system 198, and corrective action system 200. It can alsoillustratively include a calibration system 202, one or morewarning/notification generator components 204, and data store 206.

In the example shown in FIG. 6, corrective action system 200illustratively includes problem identifying component 208, correctiveaction identifying component 210, and it can include other items 212.Data store 206 can include one or more prescriptions 214, a set ofobserved values 216, one or more difference maps 218, and it can includeother items 220.

In the example described with respect to FIG. 6, mobile crop caremachine 102 is illustratively applying one or more chemicals to a field.Therefore, each prescription 214 can include a chemical type 222, achemical concentration 224, a location of application 226, and it caninclude a wide variety of other information 228.

FIG. 7 is a flow diagram illustrating one example of the operation ofarchitecture 152 shown in FIG. 3 using a crop care controller 140 suchas that described with respect to FIG. 6. It will also be noted that,instead of having both a forward flying UAV 104 and a rearward UAV 154,the same architecture can be implemented using a single UAV thatalternately flies ahead of, and behind, mobile crop care machine 102.Further, it can be implemented using multiple forward and rearward UAVs.

Mobile crop care machine 102 first receives information from forward UAV104. This is indicated by block 250 in FIG. 7. This can be carried out,for example, according to the operation described above with respect toFIG. 4. Receiving information from the forward UAV 104 can take a widevariety of different forms. It can include the sensed attribute asindicated by block 251, the location where the sensed, attribute wassensed as indicated by block 253, and it can include other items 254.

Prescription generator component 192 then generates a prescription, andmachine 102 then performs the crop care operation by controllingcontrollable mechanisms 142 based upon the information received fromforward UAV 104 (e.g., based on the prescription). This is indicated byblock 252. For example, where machine 102 is a sprayer, it can applychemicals of a certain chemical type and concentration at variouslocations, according to the prescription. Where multiple chemicals areinvolved, or multiple concentrations, each of them may have a uniquechemical marker which allows its as-applied pattern to be detected bythe attribute sensors 112 in the rearward flying UAV 154. Thus, rearwardUAV 154 senses information and provides it to mobile crop care machine102 over link 156. This is indicated by block 260 in FIG. 7.

Receiving information from rearward UAV 154 can also take a wide varietyof forms. The information can be the sensed attribute itself, asindicated by block 263, the location 265 where the sensed attribute wassensed, and a wide variety of other information 267.

Geographical correlation component 190 in crop care component 140 thencorrelates the forward and rearward information based upon theirlocations. This is indicated by block 262. For instance, it correlatesthe information acquired by forward UAV 104 with the informationacquired by rearward UAV 154, so that the information corresponds to thesame plot of ground on the field.

Difference map generation component 194 then generates a difference mapindicative of a difference between the prescribed and actual crop careoperation. This is indicated by block 264. For instance, if a particularset of chemicals at various concentrations were to be applied atdifferent locations on the field, then the difference map will indicatewhether, and how closely, the actual spraying operation conformed to theprescription.

Difference map generation component 194 then outputs the difference mapfor use. This is indicated by block 266. The difference map can be usedin a wide variety of different ways. It can be used by mobile crop caremachine 102, itself. This is indicated by block 268. For example,machine 202 may have an additional controllable mechanism 280 (such as asecond set of nozzles, that are deployed behind UAV 154 or a secondtowed sprayer or a chemical applicator). It can also be used toinfluence the operation of another mobile crop care machine. This isindicated by block 270. FIG. 8 illustrates one example of this.

It can be seen in FIG. 8 that a plurality of secondary mobile crop caremachines 272-274 are provided that follow rearward UAV 154. Machine 272is a touch-up UAV that includes a chemical applicator 276 that can beused to apply chemical to the field or worksite. Machine 274 is anotherground-traveling machine 274, again with an applicator 278 that can beused to apply chemicals. Machines 272-274 may be manned, autonomous,semi-autonomous, or additional, tethered machines.

Based on the difference map, areas 282 and 284 are identified as needingtouch-up. For example, it may be that the prescription was not followedprecisely enough with respect to areas 282 and 284. In that case, thedifference map (or some indication or metrics indicative of thedifference map) can be provided to one or more machines 272-274 that canfollow-up and spray additional chemicals on areas 282 and 284. In yetanother example, rearward UAV 152, itself, has a chemical applicator canthat can be used to treat spots 282 and 284 as well.

As one implementation, the difference map may have its vector regions ormatrix elements classified for application of chemical as “adequate”,“marginal”, “deficient”, etc. The deficient areas 282 and 284 may beidentified by the processor as having enough economic or other interestto touch up. Deficient area 282 can be assigned to a touch-up UAV 272which may apply chemical to bring the area into the “adequate”application status. Referring again to FIG. 7, the difference map mayalso be used to generate alerts or notifications for the user. This isindicated by block 290 in FIG. 7.

By way of example, it may be that a chemical marker (used to identifywhether the application conformed to the prescription) may be expensive.Thus, it may not be continuously applied across the worksite (or field).In such cases, it may be applied to a diagnostic portion of the worksiteor field, and actions can be taken based upon what occurred in thediagnostic portion. For instance, the difference map can include valuesthat are used to generate an application metric that is indicative ofthe quality of the application (e.g., how closely it conformed to theprescription). As one example, each portion 160 for which data isacquired, and for which a prescription is generated, may be divided intosub-sections, and each sub-section may have a value indicating how wellthat actual application conformed to the prescription. FIG. 8Aillustrates one example of a difference map for such a portion.

In the example shown in FIG. 8A, it is assumed that the controllablemechanism 142 is a sprayer that has 8 nozzles or sub-sections. Thus, thedifference map includes a value corresponding to each nozzle orsub-section. The value generated for each portion of the difference mapindicates whether a given chemical was over-applied, applied asprescribed, under-applied, etc. Thus, the eight values on the differencemap shown in FIG. 8A are +5, 0, +1, −1, −3, −5, −4 and −4. Thedifference values can represent a percent (or other) deviation fromprescription or another variable indicating how closely the actualapplication conformed to the prescription. The values can be used togenerate an application metric.

For instance, the application metric can be generated by summingtogether all of the individual values in the difference map for acorresponding portion of the worksite. The sum may be a simple sum, asum of absolute error values, a weighted sum, or any other metric. Oncethe metric is calculated, it can be compared to one or more alertthresholds. For instance, the alert thresholds may include a firstthreshold. If the metric is within the first threshold, then anotification can be generated indicating that the operation is beingperformed adequately. If the metric exceeds the first threshold, but iswithin a second threshold, that may indicate that the operation is beingperformed adequately, but is near the border of inadequate performance.In that case, a cautionary notification may be generated. If the metricexceeds the second threshold, this may generate a warming alertindicating that the process is being performed inadequately. Of course,there may be a wide variety of different numbers and types of thresholdsto generate a wide variety of different types of alerts ornotifications.

It should also be noted that the alerts and notifications can take awide variety of different forms. They can be provided to a localoperator or to a remote site. They can be visual communications (such ascolor-coded green, yellow, red, etc.). They can be audible (such as nosound, intermittent tone, continuous tone, etc.). They can be haptic(such as no seat or phone vibration, intermittent vibration, continuousvibration, etc.), or they can take any of a wide variety of other forms.

Referring again to FIG. 7, the difference map can be output to generatea visually observable quality map. This is indicated by block 292. Thequality map may include, for instance, a geographical representation ofthe field, and visually observable identifiers indicating the quality ofthe crop care operation, as it was performed on each of the identifiedlocations in the field. In another example, only the areas where theoperation was performed insufficiently are identified. The quality mapcan take a wide variety of other forms as well.

Referring again to FIG. 7, the difference map can be used to performother functions as well. For instance, it can be used to perform patternidentification 294 that may indicate problematic patterns. It can beused to perform error processing and error correction as indicated byblock 296, and it can be used to identify machine problems and togenerate correction indicators correcting those problems. Machineproblems may include setup problems which indicate problems with respectto the configuration or setup of the machine, or they can indicateactual machine malfunctions. All of these are indicated by block 298.

FIGS. 9 and 10 will now be described to indicate number of examples ofpattern identification 294, error processing and correction 296, andmachine problem identification and correction 298. It will be notedthat, in performing these types of processes, crop care controller 140can collect supplemental information from collection system 198. Anumber of examples of this are described below as well.

FIG. 9 shows one example of architecture 152, in which some items aresimilar to those shown in FIG. 3, and they are similarly numbered. FIG.9, however, shows that in the example discussed, machine 102 includes asprayer mechanism 300 that includes chemical 302, a distribution system304 (which can include such things as pumps, distribution lines, etc.)and a set of nozzle valve actuators 306. Controllable mechanism 142includes an array of spray nozzles 308. Each nozzle may be individuallycontrollable, or they may be controllable in segments. Nozzle valveactuators 306 control pumps and lines 304 to deliver one or morechemicals 302, in various concentrations, through each of the nozzles308 to the corresponding portion of the field to be treated. Forward UAV104 senses the attributes of portion 162 so that a prescription can begenerated for chemical delivery to that portion, when machine 102 isover it. Rearward UAV 154 senses the quality of that application.

In some examples, application error may be related to a machine orenvironmental situation for which compensation can be performed toreduce application error. In that case, the difference map can beanalyzed by corrective action system 200. For instance, problemidentifying component 208 may identify patterns that indicate problems,and corrective action identifying component 210 can identify correctiveactions that can be taken to mitigate or eliminate those problems.Components 208 and 210 can use information received from supplementalinformation collection system 198, as well.

As one example, the application error may be higher where there aresignificant cross winds. In such an example, the chemical may be blownin the direction of the cross winds, between the time it is dischargedfrom the nozzles 308 and the time it reaches the crop to be treated.Thus, supplemental information collection system 198 may be a systemthat measures the wind speed and direction. This may be measured locallyrelative to machine 102, or it may be obtained from a nearby weatherstation, or otherwise. By considering the wind speed and direction, itmay be that problem identifying component 208 and corrective actionidentifying component 210 identify that the blowing wind has spatiallyshifted the chemical from its prescribed location to a location that isdisplaced from the prescribed location by a distance that isproportional to (or otherwise related to) the wind vector. In that case,corrective action identifying component 210 may identify that acorrective action includes modifying which outlet locations (e.g., whichnozzles 308) are used to apply the chemicals, and the timing of chemicalrelease. For instance, with a certain wind vector, it may be determinedto shift nozzle assignments 50 cm in one direction, and delayapplication for 500 ms (relative to the prescription that would beapplied during a calm wind situation), for a given machine speed anddirection. This will apply the chemical with less deviation from theprescribed application. FIG. 10 (which includes FIGS. 10A, 10B and 10C)illustrates this. FIG. 10 assumes the direction of travel of the machinerelative to the illustrated portions is as shown by arrow 158.

FIG. 10A shows that, for one portion 160, the as-prescribed chemicalapplication may be represented by 310. This indicates that, for asignificant portion of the area being treated, a chemical application atrate 1 should be provided. However, for a relatively smaller portionnear the center of the overall portion, the chemical should be appliedat rate 2. When the rearward UAV 154 travels over the portion where theapplication was made, it senses a pattern of application indicated by312. This indicates that the portion of application corresponding torate 2 has shifted from the prescribed location. This (in conjunctionwith a wind vector, if one is used) may tend to indicate that the windhas affected the application. In that case, corrective action system 200may identify the problem and correct future applications to accommodatethe wind speed and direction.

FIG. 10B shows an example of a pattern that can be identified bycorrective action system 200 to identify a machine malfunction. In FIG.2B, numeral 314 indicates that the entire portion being treated shouldreceive a chemical application at rate 1. However, numeral 316 shows theas-applied rate, as sensed by rearward UAV 154. It can be seen thatthere are two sections 318 and 320 that received the application at rate1. However, there is also a section 322 that received no application.Because the direction of travel is indicated by arrow 158, section 322may correspond to a nozzle that has failed or plugged. In that case, theoperator can be notified immediately, and some indication as to thenature of the problem may be provided as well. For instance, anotification may be generated indicating that “nozzle 6 is plugged orhas failed”.

FIG. 10C also shows another pattern that can be used to identify aproblem. Numeral 324 shows that the portion being treated is prescribedto receive the chemical at rate 1. However, the as-applied pattern showsthat portion 326 received an application at rate 1, but portion 328received an application at only half of rate 1. This indicates that aspray nozzle (or section) may have a reduced flow rate, but that it hasnot failed entirely. Where the pattern indicates that a nozzle applies aconsistently low rate, relative to the prescription, then that nozzlemay be controlled to apply X % more chemical to bring the as-appliedrate closer to the prescribed rate.

In yet another example, these patterns may indicate that a pumpsupplying the chemical, or a valve regulating flow of the chemical, to agiven section nozzle may be leaking. This can result in reducedapplication from that nozzle. The pattern may thus indicate aconsistently low as-applied rate, relative to the prescription, for anozzle or for a number of nozzles that are served by the given pump orvalve. An appropriate alert or notification can be surfaced for theuser.

Referring again to the flow diagram of FIG. 7, the information generatedby UAVs 104 and 154 (or by either of them) may also be used to performcalibration operations using diagnostic zone processing. This isindicated by block 350. FIG. 11 shows one example of this. FIG. 11 showsthat, in order to reduce cost or to address other factors associatedwith a chemical marker, use of the chemical marker may be restricted toa set of diagnostic zones on the worksite. In FIG. 11, an a prioriapplication path 352 may be calculated for machine 102. However, it maybe that the machine only performs processing at a set of diagnosticzones along that path. The set of diagnostic zones may also beidentified a priori or in situ, or using a combination of both a prioriand in situ identification.

Some criteria that can be used to define the location of the diagnosticzones may include, for instance, wind direction, wind speed, applicationrate, application outlets used by prescription, etc. In FIG. 11, forinstance, diagnostic zones 354 and 356 may be used for wind calibrationwhen traveling in the East bound direction on FIG. 11. An applicationprescription can be generated for zone 354 and then the application canbe applied and the as-applied pattern can be detected. Adjustments canbe made based upon how well the as-applied pattern conforms to theprescription, and the same processing can be performed with respect tozone 356. Diagnostic zones 358 and 360 can be used for wind calibrationwhen traveling west.

In another example, diagnostic zones 362 and 364 can be used to checkhigh application rate component health and diagnostic zones 366 and 368can be used to check low application rate component health. Additionaldiagnostic zones can be assigned based on field conditions. Suchdiagnostic zones can vary widely, based on individual field conditionsand the particular crop care function being performed.

It should also be noted that the present architectures contemplate usingmachine 102 to perform multiple passes. For instance, on a first pass, asingle UAV 104 may fly forward of machine 102. In the second pass, itmay fly rearward of machine 102. In other examples, both forward andrearward UAVs are used, and the second pass of machine 104 is used toperform touch-up operations. All of these are contemplated herein.

Referring again to FIG. 7, a wide variety of post-operation analytics351 can be performed on the data described herein. It can be done onmachine 102 or sent to a remote server environment where the analyticsare performed. The data can be consumed in other ways 353 as well.

FIG. 12 shows yet another example of machine 102. Machine 102illustratively has forward UAV 104 and rearward UAV 154. They areillustratively tethered to machine 102 using physical tethers whichrepresent links 108 and 156. FIG. 12 also illustrates that machine 102is illustratively provided with a landing area (or docking area) 380. Inthe example shown in FIG. 12, docking area 380 illustratively includesan area for UAVs 104 and 154 to land or dock. They can includemechanical securing or plugging mechanisms that are automatically ormanually actuated to secure UAVs 104 and 154 to machine 102. They caninclude power or battery charging docking elements, data transmissionconnectors, or other mechanical, electronic or electromagneticcomponents as well.

It can be seen that the present system provides real time, or near realtime, control. Because the one or more UAVs provide data about theworksite at the same time that the mobile machine is operating on theworksite, this type of control can be achieved. This is true, regardlessof whether UAVs simultaneously operate forward of, and rearward of, themobile machine, or whether one UAV operates rearward of the mobilemachine or alternates between operating forward and rearward of themobile machine.

Embodiments of the present system advantageously have images withsufficient resolution in order to distinguish small items in the field.

The present discussion has mentioned processors and servers. In oneexample, the processors and servers include computer processors withassociated memory and timing circuitry, not separately shown. They arefunctional parts of the systems or devices to which they belong and areactivated by, and facilitate the functionality of the other componentsor items in those systems.

Also, a number of user interface displays have been discussed. They cantake a wide variety of different forms and can have a wide variety ofdifferent user actuatable input mechanisms disposed thereon. Forinstance, the user actuatable input mechanisms can be text boxes, checkboxes, icons, links, drop-down menus, search boxes, etc. They can alsobe actuated in a wide variety of different ways. For instance, they canbe actuated using a point and click device (such as a track ball ormouse). They can be actuated using hardware buttons, switches, ajoystick or keyboard, thumb switches or thumb pads, etc. They can alsobe actuated using a virtual keyboard or other virtual actuators. Inaddition, where the screen on which they are displayed is a touchsensitive screen, they can be actuated using touch gestures. Also, wherethe device that displays them has speech recognition components, theycan be actuated using speech commands.

A number of data stores have also been discussed. It will be noted theycan each be broken into multiple data stores. All can be local to thesystems accessing them, all can be remote, or some can be local whileothers are remote. All of these configurations are contemplated herein.

Also, the figures show a number of blocks with functionality ascribed toeach block. It will be noted that fewer blocks can be used so thefunctionality is performed by fewer components. Also, more blocks can beused with the functionality distributed among more components.

It will also be noted that the difference map or any of the otherinformation can be output to a remote server environment.

FIG. 13 is a block diagram of machine 102, shown in FIG. 1, except thatit communicates with elements in a remote server architecture 500. In anexample, remote server architecture 500 can provide computation,software, data access, and storage services that do not require end-userknowledge of the physical location or configuration of the system thatdelivers the services. In various embodiments, remote servers candeliver the services over a wide area network, such as the internet,using appropriate protocols. For instance, remote servers can deliverapplications over a wide area network and they can be accessed through aweb browser or any other computing component. Software or componentsshown in the previous Figures as well as the corresponding data, can bestored on servers at a remote location. The computing resources in aremote server environment can be consolidated at a remote data centerlocation or they can be dispersed. Remote server infrastructures candeliver services through shared data centers, even though they appear asa single point of access for the user. Thus, the components andfunctions described herein can be provided from a remote server at aremote location using a remote server architecture. Alternatively, theycan be provided from a conventional server, or they can be installed onclient devices directly, or in other ways.

In the example shown in FIG. 13, some items are similar to those shownin previous Figures and they are similarly numbered. FIG. 13specifically shows that crop care controller (or portions of it) anddata store 206 can be located at a remote server location 502.Therefore, machine 102 accesses those systems through remote serverlocation 502.

FIG. 13 also depicts another example of a remote server architecture.FIG. 13 shows that it is also contemplated that some elements aredisposed at remote server location 502 while others are not. By way ofexample, data store 206 or a third party system 507 can be disposed at alocation separate from location 502, and accessed through the remoteserver at location 502. Other parts of the machine 102 (e.g., parts ofcontrol system 140) can be stored at remote server location 502 orelsewhere. Regardless of where they are located, they can be accesseddirectly by machine 102, or user 508 through a network (either a widearea network or a local area network), they can be hosted at a remotesite by a service, or they can be provided as a service, or accessed bya connection service that resides in a remote location. Also, the datacan be stored in substantially any location and intermittently accessedby, or forwarded to, interested parties. For instance, physical carrierscan be used instead of, or in addition to, electromagnetic wavecarriers. In such an embodiment, where cell coverage is poor ornonexistent, another mobile machine (such as a fuel truck) can have anautomated information collection system. As machine 102 (or any UAVs)comes close to the fuel truck for fueling, the system automaticallycollects the information from the machine (or UAV) using any type ofad-hoc wireless connection. The collected information can then beforwarded to the main network as the fuel truck reaches a location wherethere is cellular coverage (or other wireless coverage). For instance,the fuel truck may enter a covered location when traveling to fuel othermachines or when at a main fuel storage location. All of thesearchitectures are contemplated herein. Further, the information can bestored on machine 102 until machine 102 enters a covered location. Themachine 102, itself, can then send the information to the main network.

It will also be noted that the elements of FIG. 1, or portions of them,can be disposed on a wide variety of different devices. Some of thosedevices include servers, desktop computers, laptop computers, tabletcomputers, or other mobile devices, such as palm top computers, cellphones, smart phones, multimedia players, personal digital assistants,etc.

FIG. 14 is a simplified block diagram of one illustrative example of ahandheld or mobile computing device that can be used as a user's orclient's hand held device 16, in which the present system (or parts ofit) can be deployed. For instance, a mobile device can be deployed inthe operator compartment of machine 102 for use in generating,processing, or displaying the data. FIGS. 15-16 are examples of handheldor mobile devices.

FIG. 14 provides a general block diagram of the components of a clientdevice 16 that can run some components shown in previous Figures, thatinteracts with them, or both. In the device 16, a communications link 13is provided that allows the handheld device to communicate with othercomputing devices and under some embodiments provides a channel forreceiving information automatically, such as by scanning. Examples ofcommunications link 13 include allowing communication though one or morecommunication protocols, such as wireless services used to providecellular access to a network, as well as protocols that provide localwireless connections to networks.

In other examples, applications can be received on a removable SecureDigital (SD) card that is connected to an interface 15. Interface 15 andcommunication links 13 communicate with a processor 17 (which can alsoembody processors or servers from previous Figures) along a bus 19 thatis also connected to memory 21 and input/output (I/O) components 23, aswell as clock 25 and location system 27.

I/O components 23, in one embodiment, are provided to facilitate inputand output operations. I/O components 23 for various embodiments of thedevice 16 can include input components such as buttons, touch sensors,optical sensors, microphones, touch screens, proximity sensors,accelerometers, orientation sensors and output components such as adisplay device, a speaker, and or a printer port. Other I/O components23 can be used as well.

Clock 25 illustratively comprises a real time clock component thatoutputs a time and date. It can also, illustratively, provide timingfunctions for processor 17.

Location system 27 illustratively includes a component that outputs acurrent geographical location of device 16. This can include, forinstance, a global positioning system (GPS) receiver, a LORAN system, adead reckoning system, a cellular triangulation system, or otherpositioning system. It can also include, for example, mapping softwareor navigation software that generates desired maps, navigation routesand other geographic functions.

Memory 21 stores operating system 29, network settings 31, applications33, application configuration settings 35, data store 37, communicationdrivers 39, and communication configuration settings 41. Memory 21 caninclude all types of tangible volatile and non-volatilecomputer-readable memory devices. It can also include computer storagemedia (described below). Memory 21 stores computer readable instructionsthat, when executed by processor 17, cause the processor to performcomputer-implemented steps or functions according to the instructions.Processor 17 can be activated by other components to facilitate theirfunctionality as well.

FIG. 15 shows one example in which device 16 is a tablet computer 600.In FIG. 16, computer 600 is shown with user interface display screen602. Screen 602 can be a touch screen or a pen-enabled interface thatreceives inputs from a pen or stylus. It can also use an on-screenvirtual keyboard. Of course, it might also be attached to a keyboard orother user input device through a suitable attachment mechanism, such asa wireless link or USB port, for instance. Computer 600 can alsoillustratively receive voice inputs as well.

Additional examples of devices 16 can be used as well. Device 16 can be,a feature phone, smart phone or mobile phone. The phone can include aset of keypads for dialing phone numbers, a display capable ofdisplaying images including application images, icons, web pages,photographs, and video, and control buttons for selecting items shown onthe display. The phone can include an antenna for receiving cellularphone signals such as General Packet Radio Service (GPRS) and 1Xrtt, andShort Message Service (SMS) signals. In some examples the phone alsoincludes a Secure Digital (SD) card slot that accepts a SD card.

The mobile device can also be a personal digital assistant or amultimedia player or a tablet computing device, etc. (hereinafterreferred to as a PDA). The PDA can include an inductive screen thatsenses the position of a stylus (or other pointers, such as a user'sfinger) when the stylus is positioned over the screen. This allows theuser to select, highlight, and move items on the screen as well as drawand write. The PDA can also include a number of user input keys orbuttons which allow the user to scroll through menu options or otherdisplay options which are displayed on the display, and allow the userto change applications or select user input functions, withoutcontacting the display. The PDA can also include an internal antenna andan infrared transmitter/receiver that allow for wireless communicationwith other computers as well as connection ports that allow for hardwareconnections to other computing devices. Such hardware connections aretypically made through a cradle that connects to the other computerthrough a serial or USB port. As such, these connections are non-networkconnections.

FIG. 16 shows that the phone is a smart phone 71. Smart phone 71 has atouch sensitive display 73 that displays icons or tiles or other userinput mechanisms 75. Mechanisms 75 can be used by a user to runapplications, make calls, perform data transfer operations, etc. Ingeneral, smart phone 71 is built on a mobile operating system and offersmore advanced computing capability and connectivity than a featurephone.

Note that other forms of the devices 16 are possible.

FIG. 17 is one example of a computing environment in which elements ofthe previous Figures, or parts of them, (for example), can be deployed.With reference to FIG. 17, an example system for implementing someembodiments includes a general-purpose computing device in the form of acomputer 810. Components of computer 810 may include, but are notlimited to, a processing unit 820 (which can comprise processors orservers from previous Figures), a system memory 830, and a system bus821 that couples various system components including the system memoryto the processing unit 820. The system bus 821 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. Memory and programs described with respect to previousFigures can be deployed in corresponding portions of FIG. 17.

Computer 810 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 810 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media is different from, anddoes not include, a modulated data signal or carrier wave. It includeshardware storage media including both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by computer 810. Communication media may embody computerreadable instructions, data structures, program modules or other data ina transport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal.

The system memory 830 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 831and random access memory (RAM) 832. A basic input/output system 833(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 810, such as during start-up, istypically stored in ROM 831. RAM 832 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 820. By way of example, and notlimitation, FIG. 17 illustrates operating system 834, applicationprograms 835, other program modules 836, and program data 837.

The computer 810 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 17 illustrates a hard disk drive 841 that reads from or writes tonon-removable, nonvolatile magnetic media, an optical disk drive 855,and nonvolatile optical disk 856. The hard disk drive 841 is typicallyconnected to the system bus 821 through a non-removable memory interfacesuch as interface 840, and optical disk drive 855 is typically connectedto the system bus 821 by a removable memory interface, such as interface850.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Application-specific Integrated Circuits (e.g., ASICs),Application-specific Standard Products (e.g., ASSPs), System-on-a-chipsystems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 17, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 810. In FIG. 17, for example, hard disk drive 841 isillustrated as storing operating system 844, application programs 845,other program modules 846, and program data 847. Note that thesecomponents can either be the same as or different from operating system834, application programs 835, other program modules 836, and programdata 837.

A user may enter commands and information into the computer 810 throughinput devices such as a keyboard 862, a microphone 863, and a pointingdevice 861, such as a mouse, trackball or touch pad. Other input devices(not shown) may include a joystick, game pad, satellite dish, scanner,or the like. These and other input devices are often connected to theprocessing unit 820 through a user input interface 860 that is coupledto the system bus, but may be connected by other interface and busstructures. A visual display 891 or other type of display device is alsoconnected to the system bus 821 via an interface, such as a videointerface 890. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 897 and printer 896,which may be connected through an output peripheral interface 895.

The computer 810 is operated in a networked environment using logicalconnections (such as a local area network—LAN, or wide area network WAN)to one or more remote computers, such as a remote computer 880.

When used in a LAN networking environment, the computer 810 is connectedto the LAN 871 through a network interface or adapter 870. When used ina WAN networking environment, the computer 810 typically includes amodem 872 or other means for establishing communications over the WAN873, such as the Internet. In a networked environment, program modulesmay be stored in a remote memory storage device. FIG. 17 illustrates,for example, that remote application programs 885 can reside on remotecomputer 880.

It should also be noted that the different embodiments described hereincan be combined in different ways. That is, parts of one or moreembodiments can be combined with parts of one or more other embodiments.All of this is contemplated herein.

Example 1 is a mobile machine, comprising:

-   -   a controllable mechanism that performs an operation on a        worksite as the mobile machine travels over the worksite in a        direction of travel;    -   a communication system that receives attribute data indicative        of a sensed attribute of a location of the worksite, and that        receives effect data indicative of an effect of the operation on        the location of the worksite rearward of the mobile machine in        the direction of travel, after the controllable mechanism has        performed the operation at the location, and the communication        system receiving the effect data from a first unmanned aerial        vehicle (UAV), over a communication link between the first UAV        and the mobile machine; and    -   a control system that controls the controllable mechanism based        on the attribute data.

Example 2 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a geographical correlation component that correlates the        attribute data and the effect data to the location.

Example 3 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a prescription generator component that generates a prescribed        operation for the controllable mechanism based on the attribute        data, the control system controlling the controllable mechanism        based on the prescribed operation.

Example 4 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a difference map generation component that determines a        difference between the prescribed operation at the location and        the operation performed at the location for a plurality of        locations in the worksite, and generates a difference map        correlating the determined differences to the plurality of        locations.

Example 5 is the mobile machine of any or all previous examples whereinthe control system generates an action signal to take a correctiveaction to address differences on the difference map.

Example 6 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a pattern identifier component that identifies a pattern of        differences based on the difference map and generates a pattern        signal indicative of the identified pattern.

Example 7 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a problem identifying component that receives the pattern signal        and identifies a problem based on the identified pattern; and    -   a corrective action identifying component that identifies a        corrective action based on the identified problem.

Example 8 is the mobile machine of any or all previous examples whereinthe problem identifying component identifies machine problems with themobile machine and the controllable mechanism.

Example 9 is the mobile machine of any or all previous examples whereinthe control system further comprises:

-   -   a notification generator component that generates an operator        notification based on the identified problem.

Example 10 is the mobile machine of any or all previous examples andfurther comprising:

-   -   a calibration system that generates a calibration signal based        on a given set of locations on the difference map, the control        system calibrating at least one of the mobile machine and the        controllable mechanism based on the calibration signal.

Example 11 is the mobile machine of any or all previous examples whereinthe first UAV flies rearward of the mobile machine in the direction oftravel and wherein the attribute data is received from a second UAV thatflies forward of the mobile machine in the direction of travel, andfurther comprising:

-   -   a first physical tether that tethers the first UAV to the mobile        machine; and a second physical tether that tethers the second        UAV to the mobile machine.

Example 12 is the mobile machine of any or all previous examples whereinthe first and second UAVs each include an image capture device thatcaptures an image of the location to generate the attribute data andeffect data, respectively.

Example 13 is the mobile machine of any or all previous examples whereinthe control system sends the action signal to a second mobile machine toperform a follow-up operation at locations on the difference map.

Example 14 is a computer implemented method, comprising:

-   -   receiving attribute data indicative of a sensed attribute of a        location of a worksite, forward of a mobile machine in the        direction of traveler,    -   generating a prescribed operation indicator, indicative of a        prescribed operation to perform at the location, based on the        attribute data;    -   controlling a controllable mechanism, coupled to the mobile        machine, to perform the prescribed operation at the location of        the worksite, based on the prescribed operation indicator;    -   receiving, over a communication link, from a first unmanned        aerial vehicle (UAV), effect data indicative of an effect of the        operation on the location of the worksite after the controllable        mechanism has performed the operation at the location; and    -   generating an action signal to perform an action based on the        effect data.

Example 15 is the computer implemented method of any or all previousexamples wherein generating a prescribed operation indicator comprises:

-   -   correlating the attribute data to the location; and    -   providing the location along with the prescribed operation        indicator.

Example 16 is the computer implemented method of any or all previousexamples wherein generating an action signal comprises:

-   -   determining a difference between the prescribed operation at the        location and the operation performed at the location, for a        plurality of locations in the worksite;    -   generating a difference map correlating the determined        differences to the plurality of locations; and    -   generating the action signal to take a corrective action to        address differences on the difference map.

Example 17 is the computer implemented method of any or all previousexamples wherein generating the action signal further comprises:

-   -   identifying a pattern of differences based on the difference        map;    -   generating a pattern signal indicative of the identified        pattern;    -   identifying a problem based on the pattern signal;    -   identifying a corrective action based on the identified problem;        and    -   generating the action signal to take the corrective action.

Example 18 is a mobile machine system, comprising:

-   -   a first unmanned aerial vehicle (UAV);    -   a second UAV; and    -   a mobile machine, comprising:    -   a controllable mechanism that performs an operation on a        worksite, as the mobile machine moves over the worksite in a        direction of travel; and    -   a control system that receives, from the first UAV, attribute        data indicative of an attribute of the worksite sensed by the        first UAV at a location forward of the mobile machine in the        direction of travel, the control system generating a prescribed        operation to perform at the location and controlling the        controllable mechanism to perform the operation at the location        based on the prescribed operation, the control system receiving        effect data from the second UAV indicative of an effect of the        operation performed at the location after the operation has been        performed at the location, and generating an action signal based        on differences between the prescribed operation and the        operation performed at the location.

Example 19 is the mobile machine system of any or all previous exampleswherein the first and second UAVs each have an image capture sensor thatcaptures an image of the location, the attribute data and the effectdata being indicative of the images.

Example 20 is the mobile machine system of any or all previous exampleswherein at least one of the first and second UAVs are physicallytethered to the mobile machine by physical tethers.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A mobile machine, comprising: a controllablemechanism that performs an operation on a worksite as the mobile machinetravels over the worksite in a direction of travel; a communicationsystem that receives attribute data indicative of a sensed attribute ofa location of the worksite, and that receives effect data indicative ofan effect of the operation on the location of the worksite rearward ofthe mobile machine in the direction of travel, after the controllablemechanism has performed the operation at the location, the communicationsystem receiving the effect data from a first unmanned aerial vehicle(UAV), over a communication link between the first UAV and the mobilemachine; and a control system that controls the controllable mechanismbased on the attribute data.
 2. The mobile machine of claim 1 whereinthe control system further comprises: a geographical correlationcomponent that correlates the attribute data and the effect data to thelocation.
 3. The mobile machine of claim 2 wherein the control systemfurther comprises: a prescription generator component that generates aprescribed operation for the controllable mechanism based on theattribute data, the control system controlling the controllablemechanism based on the prescribed operation.
 4. The mobile machine ofclaim 3 wherein the control system further comprises: a difference mapgeneration component that determines a difference between the prescribedoperation at the location and the operation performed at the locationfor a plurality of locations in the worksite, and generates a differencemap correlating the determined differences to the plurality oflocations.
 5. The mobile machine of claim 4 wherein the control systemgenerates an action signal to take a corrective action to addressselected differences on the difference map.
 6. The mobile machine ofclaim 4 wherein the control system further comprises: a patternidentifier component that identifies a pattern of differences based onthe difference map and generates a pattern signal indicative of theidentified pattern.
 7. The mobile machine of claim 6 wherein the controlsystem further comprises: a problem identifying component that receivesthe pattern signal and identifies a problem based on the identifiedpattern; and a corrective action identifying component that identifies acorrective action based on the identified problem.
 8. The mobile machineof claim 7 wherein the problem identifying component identifies machineproblems with the controllable mechanism.
 9. The mobile machine of claim7 wherein the control system further comprises: a notification generatorcomponent that generates an operator notification based on theidentified problem.
 10. The mobile machine of claim 4 and furthercomprising: a calibration system that generates a calibration signalbased on a given set of locations on the difference map, the controlsystem calibrating at least one of the mobile machine and thecontrollable mechanism based on the calibration signal.
 11. The mobilemachine of claim 1 wherein the first UAV flies rearward of the mobilemachine in the direction of travel and wherein the attribute data isreceived from a second UAV that flies forward of the mobile machine inthe direction of travel, and further comprising: a first physical tetherthat tethers the first UAV to the mobile machine.
 12. The mobile machineof claim 11 and further comprising: a second physical tether thattethers the second UAV to the mobile machine.
 13. The mobile machine ofclaim 11 wherein the first and second UAVs each include an image capturedevice that captures an image of the location to generate the attributedata and the effect data, respectively.
 14. The mobile machine of claim5 wherein the control system sends the action signal to a second mobilemachine to perform a follow-up operation at selected locations on thedifference map.
 15. A computer implemented method, comprising: receivingattribute data indicative of a sensed attribute of a location of aworksite, forward of a mobile machine in the direction of travel,generating a prescribed operation indicator, indicative of a prescribedoperation to perform at the location, based on the attribute data;controlling a controllable mechanism, coupled to the mobile machine, toperform the prescribed operation at the location of the worksite, basedon the prescribed operation indicator; receiving, over a communicationlink, from a first unmanned aerial vehicle (UAV), effect data indicativeof an effect of the operation on the location of the worksite after thecontrollable mechanism has performed the operation at the location; andgenerating an action signal to perform an action based on the effectdata.
 16. The computer implemented method of claim 15 wherein generatinga prescribed operation indicator comprises: correlating the attributedata to the location; and providing the location along with theprescribed operation indicator.
 17. The computer implemented method ofclaim 16 wherein generating an action signal comprises: determining adifference between the prescribed operation at the location and theoperation performed at the location, for a plurality of locations in theworksite; generating a difference map correlating the determineddifferences to the plurality of locations; and generating the actionsignal to take a corrective action to address differences on thedifference map.
 18. The computer implemented method of claim 17 whereingenerating the action signal further comprises: identifying a pattern ofdifferences based on the difference map; generating a pattern signalindicative of the identified pattern; identifying a problem based on thepattern signal; identifying a corrective action based on the identifiedproblem; and generating the action signal to take the corrective action.19. A mobile machine system, comprising: a first unmanned aerial vehicle(UAV); a second UAV; and a mobile machine, comprising: a controllablemechanism that performs an operation on a worksite, as the mobilemachine moves over the worksite in a direction of travel; and a controlsystem that receives, from the first UAV, attribute data indicative ofan attribute of the worksite sensed by the first UAV at a locationforward of the mobile machine in the direction of travel, the controlsystem generating a prescribed operation to perform at the location andcontrolling the controllable mechanism to perform the operation at thelocation based on the prescribed operation, the control system receivingeffect data from the second UAV indicative of an effect of the operationperformed at the location after the operation has been performed at thelocation, and generating an action signal based on differences betweenthe prescribed operation and the operation performed at the location.20. The mobile machine system of claim 19 wherein the first and secondUAVs each have an image capture sensor that captures an image of thelocation, the attribute data and the effect data being indicative of theimages.